Visual Object Tracking for Unmanned Aerial Vehicles: A Benchmark and New Motion Models

Despite recent advances in the visual tracking community, most studies so far have focused on the observation model. As another important component in the tracking system, the motion model is much less well-explored especially for some extreme scenarios. In this paper, we consider one such scenario in which the camera is mounted on an unmanned aerial vehicle (UAV) or drone. We build a benchmark dataset of high diversity, consisting of 70 videos captured by drone cameras. To address the challenging issue of severe camera motion, we devise simple baselines to model the camera motion by geometric transformation based on background feature points. An extensive comparison of recent state-of-the-art trackers and their motion model variants on our drone tracking dataset validates both the necessity of the dataset and the effectiveness of the proposed methods. Our aim for this work is to lay the foundation for further research in the UAV tracking area. Introduction Visual tracking is a fundamental problem pertinent to many real-world applications including video surveillance, autonomous vehicle navigation, human-computer interaction, and many more. Given the initial state (e.g., position and size) of the target object in a video frame, the goal of tracking is to automatically estimate the states of the moving object in subsequent frames. Although visual tracking has been studied for decades, it remains a challenging problem due to various factors such as partial occlusion, fast and abrupt object motion, illumination changes, and large variations in viewpoint and pose. In recent years, we have witnessed the advent of a new type of robot, unmanned aerial vehicles (UAVs) or drones (Floreano and Wood 2015). Although drones were mostly used for military applications in the past, the recent commercial drone revolution has seen an increasing number of research laboratories working on small, affordable, human-friendly drones. The rapid development of commercial drones could have a major impact on many civilian applications, including transportation and communication. Meanwhile, a number of foreseeable applications on this new platform will need visual tracking as a core enabling technology. To name a few, visual tracking can make drones Copyright c © 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. useful for tracking animals, finding people, monitoring realtime traffic situations, and so on. In this paper, we study visual tracking on the drone platform. Besides the research issues common to visual tracking in general, a major new challenge that we have to face is the abrupt camera motion frequently encountered when using drones to capture video. Specifically, a small perturbation such as a slight rotation of the camera often leads to large displacement of the target position in the image scene. Also, since a drone flies, its motion typically has a higher degree of freedom than that of many conventional tracking applications. Therefore a more sophisticated motion model is needed. As a result, conventional motion models used for tracking applications with stationary or slow-moving cameras are no longer applicable. One focus of this paper is in conducting a benchmark evaluation and proposing baseline algorithms to explicitly estimate the ego-motion. The goals of this paper are three-fold: 1. Construct a unified drone tracking benchmark dataset with detailed analysis of statistics; 2. Design general baseline algorithms for camera motion estimation and integrate them into various tracking systems; 3. Conduct an extensive experimental comparison and provide basic insights into the motion model in tracking, with the aim of opening up a new research direction for the visual tracking community.